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Abstract: The paper discussed the difficulties and problems in experimental researches in educational 
technology, such as misunderstanding in teachers’ concept, disjointedness between theory and practice, inadequate 
understanding of dialectics in experimental educational technology researches, research project selection and 
theoretical hypothesis formation are passive and onefold, single and irregular experimental control, simple ways 
and methods of measurement, more quantitative analysis and fewer qualitative analysis, simple statistical analysis. 
And provide some suggestions: bottom-up selection; various kinds of research training; emphasizing experimental 
data; controlling experimental process effectively; combination of qualitative and quantitative researches; 
integration of various research methods. 
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Educational technology is a cross-border science. Various theories in this field originate from achievements 
in other subjects ranging from informatics, psychology, sociology to philosophy. Therefore, a new tendency draws 
a great attention of more and more researchers. This new tendency is that in order to solve problems about 
psychology, education, and behavior in educational technology, investigations of phenomena and principles in 
educational technology application should be conducted across different subjects. 

Experimental research methods in educational technology is a way to collect and handle research materials 
by using mathematical tools, which are one of the important research methods to carry out educational research 
activities. In the actual educational technology research, selecting appropriate research methods has a significant 
effect on research findings. Reasonable and advanced experimental research methods can describe educational 
phenomena and disciplines in an overall, objective and accurate way, exert effectiveness of data and materials to 
the extreme, and go deep to know the disciplines behind the complicated phenomena in educational technology. 
The more experimental researches are conducted, the more mature and scientific the development of educational 
technology is. 

Experimental researches in educational technology and general experimental researches have something in 
common, and also have some differentiations. As educational researches go in depth, it becomes more essential to 
adopt mathematical analysis. Quantitative method is seen as important tool and expression method to study 
educational phenomena and discover educational disciplines. For instance, first, more and more educational 
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researches adopt multianalysis. Second, quantitative method such as “Meta-analysis” begins to be used in the 
fields that qualitative method traditionally predominates. Vague mathematics turns to be widespread in 
educational researches. Nowadays, a considerable part of research articles about educational study and 
educational technology study published in both foreign and domestic journals adopt quantitative method. 
Educational technology researchers must have a good command of quantitative method so that they can read 
relevant literatures and write high-level research articles. However in terms of present practical research articles in 
academic journals, few research articles have experimental data or evidence and necessary qualitative study. In the 
following we show our opinions about difficulties, problems and countermeasures in experimental researches in 
educational technology. 

1. Difficulties and problems in educational technology experimental researches 

1.1 Misunderstanding in teachers’ concept 

Quite a lot people in the society, including many teachers argue that the teacher is not a knowledge creator 
but a knowledge deliverer. With this view in mind, they will not see educational activities as objects of academic 
researches. Naturally they will not realize that teachers carry a heavy burden of develop educational science. In 
the aspect of developing educational academic researches, people usually lay an obligation on educators, hoping 
that they will discover new principles, create new methods, guide educational practice and develop educational 
science. 

The present situation in educational technology researches is as follows. (1) Teachers regard educational 
technology researches as jobs of theoretical workers in educational technology. Their work is instruction that has 
nothing to do with educational technology researches. (2) Although some teachers have a desire to do the 
researches, they are plunged in heavy teaching work and spare no time and effort to conduct the researches. (3) 
Still some teachers who are willing to do the researches have time and effort but they have trouble in finding 
educational technology research projects and in addition to be unfamiliar with the process and method of 
educational technology. Eventually they are frightened to flinch. 

1.2 Disjointedness between theory and practice 

In the field of educational technology, theory and practice are separated for a long time. The research projects 
of some researchers severely disjoint the actual need in educational technology. Furthermore, they generalize their 
research findings to the practice of educational technology, which teachers and educational administers can’t 
accept. Therefore, there exists a huge gap between educational technology theory and practice. Theory and 
practice are like two parallels which seem impossible to intersect forever. In order to bridge the gap, efforts from 
both educational technology theory and practice are needed. Theoretical researchers should set up the idea to serve 
the educational practice, go further into the front line of education, investigate the educational practice and help 
practical workers to settle some theoretical problems which need to be clarified. Practical workers should establish 
active mode to be diligent in discovery, cherish creative ideas and be good at finding research projects. 

1.3 Inadequate understanding of dialectics in experimental educational technology researches 

Most people simply believe that if there are adequate hardware equipments and resources, educational 
technology experiments can be carried out and surely can be successful. Most teachers and educational technology 
workers haven’t realized that experimental researches not only rely on hardware and resources, but also count on 
detailed experimental design which can reify the theoretical hypothesis. Any fault in the experimental process will 
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result in failure. But of all research articles we have read, few have negative analysis. Most research articles turn a 
blind eye to the failure of testing the hypothesis. In these experimental reports, some researchers don’t analyze 
results dialectically. Not negative results but positive results are mentioned and not failure but success is discussed. 
They never know that it is failure that contains the most valuable factors. 

1.4 Research project selection and theoretical hypothesis formation are passive and onefold 

Research projects are supposed to originate from the actual need of practitioners. Researches are expected to 

be done under the cooperation of practitioners and researchers in the practice. Research results should be 
understood, controlled and carried out. Researches should aim at solving practical problems and improving social 
activities. On the basis of the above view, action research catch greater attention of educational technology 
researchers. However, currently most our research projects selection comes from exterior, which is not generated 
from teaching practice and is passive. The flaw is to consider research hypothesis as the starting point of 
researches. 

Moreover, there are more repetitive researches and few innovative researches. In early 1980s, computers 
entered into the American primary and high schools and in China computers are also introduced into primary and 
high schools of China simultaneously. But in fact, the application and development of computer technology at 
schools depress those who expect this new technology will bring a great educational revolution. The effects of 
some computer software are not satisfactory. From 1996, the United States make great effort to develop internet 
-based online education and set up campus intranet at primary and high schools. At the end of 1990s, campus 
intranet of Chinese primary and high school arise quickly. But majority of campus intranet bring practical help to 
teaching and administration due to the lack of relevant researches and policy supports. While in 1999 and 2000 
American educational technology standards, related techniques and criteria of teachers and students are explicitly 
prescribed. That is to say, Americans invest in hardware as well as construct human resource. When we bring in 
advanced foreign teaching modes and methods, we often adopt them without any adjustment, regardless of 
historical and cultural differences among countries. Of course, this issue has already been noticed. But this also 
indicates that some of our experimental researches follow foreign researches blindly and do a lot of repetitive 
researches. Some researches imitate foreign theories or involve simple criticism so that creative ideas can not be 
generated and new ideas and methods of experimental researches can not be proposed. Consequently, academic 
experimental researches fall behind the social development. With the introduction of new concept and technology, 
educational technology researchers may probably get lost. On the other hand, the generalization of research 
findings and external validity are inadequate. Researches about teaching modes, which are not generalized into 
pedagogic practice, are relatively more but in the surface level. Research variables and methods are comparatively 
simple, not systematic. The fact that some important variables are neglected and other meaning information is 
omitted makes it difficult to draw an integrated and accurate conclusion. 

1.5 Single and irregular experimental control 

Experimental control is not normative and formalized. Experimental plan and design are rough due to 
carelessness and limited time. Independent variables of the experiments can not be manipulated, dependent 
variables can not be measured, and uncorrelated variables are not effectively controlled. Researchers have no idea 
about the experimental stage and time, and which person is responsible for a certain step, and the expected effects. 
Thus the experiment results can not meet the demand of solving educational problems. In addition, the 
standardization of tracing the experiments is low, the subject selection is at will, and simple experimental methods 
are used such as experimental group and control group. All these factors cause the low internal validity of 
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experimental researches, which possibly results in the impractical research conclusions. 

1.6 Simple ways and methods of measurement 

Measurement refers to conducting measurement on experimental environment, subjects and results by 
making use of some measurement tools in teaching process or educational experiments, in order to obtain the 
needed proof requested by hypothesis testing. However, the current measurement methods adopted in our 
experimental educational technology researches are so limited such as simple questionnaire survey and test. Few 
measurement tables are designed to examine students’ development such as thinking development and emotional 
change. It is not precise in sampling design, measurement tool selection, and evaluation. The major cause of these 
problems is that researchers lack training of using and evaluating educational measurement tools. 

1.7 More quantitative analysis and fewer qualitative analysis 

Reasons of generation and development of educational technology phenomena are complicated. Answers to 
some questions can be provided through data collection, analysis and testing. Moreover, statistical analysis of 
these data can provide a clear formalized description for educational technology researches. This situation makes 
researchers adopt more quantitative analysis and fewer qualitative analysis when they do their educational 
technology researches. Qualitative researches focus on situational study, emphasize contexts, interpret the results 
and count on context of data collection. Qualitative researches can be conducted within a school, a class, a family 
or a certain natural context. Researchers observe the behaviors and activities within their natural context and 
describe these in their entirety from beginning to end. Qualitative researches are interactions between researchers 
and subjects, which interpret the “quality” of subjects in their entirety through long, deep and detailed experience. 
In qualitative researches, it takes researchers quite a lot of time to go into schools, families and society and 
investigate related problems. Away from the context, it is impossible to know connotation and significance of 
educational activities. As a result of eagerness to obtain research findings, it should be worried about time 
consumption and finance, simple and objective quantitative data analysis influenced by advance in information 
science, most current research articles tend to use quantitative research methods. Therefore, qualitative researches 
become rare. Although some teachers make some progress through their own efforts, their achievements are often 
characterized as experimentalism and unilateralism, low theoretical standard, lack of synchronic and diachronic 
comparison, and plain analysis of problems. 

1.8 Simple statistical analysis 

Selection of statistical methods is the key factor which affects the validity of statistical results. Without 
adequate fundamental knowledge of statistics, effective statistical analysis of experimental data can not be 
achieved, which makes data analysis inaccurate and simple. Statistical methods such as X"-test, t-test, correlation 
analysis, regression analysis, multivariate analysis are seldom used by researchers. 

2. Suggestions on developing experimental researches in educational technology 

2.1 Project selection: Bottom-up selection 

Most educational experiments including educational technology experiments are carried out by individual 
schools and universities. It is school authority that makes a decision or choice about the kind of experimental 
projects, which lack communications among schools and communications between the upper and the lower. The 
school authority gets to know a certain message of educational technology experiment from a journal or a 
symposium, and they are seized by a whim to do the research without feasibility reasoning only to be in fashion. 
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Some school authorities take educational technology experiments as a way to gain reputation of emphasizing 
scientific researches, obtaining promotion or dealing with examination of the higher-up, which results in lack of 
overall picture of project selection. Therefore, the process of experiments should be more scientific. Researchers 
need to hunt for projects in educational practice. Projects need to contain explicit hypothesis. A formal selection of 
projects should be achieved after reviewing literatures carefully and reasoning. This is what bottom-up project 
selection really means. In addition, in order to study the project better, project study group need to be set up 
among different schools so that project researches can be done through cooperation by exerting power of each 
individual school. 

2.2 Various kinds of research training 

Training can be divided into two categories. On the one hand, researchers are trained by means of seminars 
and symposiums; On the other hand, methods of doing researches are taught systematically including 
experimental theory, variables, procedures, testing, and special measures in order to make sure that each group 
members involved in the research project grasp the factors and keys of experimental methods. 

2.3 Emphasizing experimental data 

In the educational technology experiments, experimental records need to be objective. The selection of 
statistical data analysis needs to be suitable. When data can not root for the experimental hypothesis, the more 
valuable factors and unexpected results may be acquired by attributing to the right reasons, finding out the failure 
factors and analyzing deeply. Hence, no matter whether an experiment fails or succeeds, it is a meaningful 
attempt. 

2.4 Controlling experimental process effectively 

Educational experiments require more labor and constrained by experimental equipments and other 
conditions so that it is difficult to control phenomena and environment. Meanwhile, subjects of educational 
experiments are man so it is of great difficulty to control the process as strictly as that in natural science. In spite 
of that, researchers should have an explicit description of experimental process and control those manmade 
intervening variables, which may have a certain effect on experimental results. Therefore, such experiments have 
requirements on researchers or all the members attending the experiments. For example, some experiments require 
researchers who are proficient in related experimental techniques and well-trained. Some experiments need the 
assistantship among schools and related units. All in all, it is essential to ensure internal and external validity. 

2.5 Combination of qualitative and quantitative researches 

Qualitative and quantitative researches are not sharply separated but relying on each other, penetrating into 
each other, and complementary to each other. In fact, qualitative researches contain elements of empirical study. 
Qualitative and quantitative researches reflect the dialectic relationship between quality and quantity of an object. 
Quality and quantity of any object are always indispensable and unitive. Quality is based on quantity and quantity 
is of a certain quality. Qualitative study on an object may naturally lead to quantitative study of this object. 
Therefore, qualitative study and quantitative study are complementary to each other. 

In the field of educational technology, quantitative and qualitative data analyses are two major domains. 
Qualitative and quantitative researches have their own characteristics and strengths. Researchers should not have 
prejudice on either of them and they should choose suitable research methods according to their own specialty and 
purposes. Combining qualitative and quantitative methods enable researchers to compare and test the results 
gained from two methods so that results could be more reliable. 

2.6 Integration of various research methods 
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Every method has its own strength and weakness. Literature review, surveys, and content analysis are 
synchronic under most circumstances but experimental research methods are generally diachronic. In order to 
acquire relatively comprehensive data and materials, it is advisable to use various methods synthetically, such as 
questionnaire, interview, observation, literature study, case study, etc. 

Take educational technology practice into consideration, we should analyze the problems and difficulties 
existed in the present experimental researches in educational technology and provide some suggestions which may 
have some limitations. In order to improve the development of experimental researches in educational technology 
and push forward educational technology, it is expected that fellow researchers in educational technology could 
exchange ideas. 

References: 

LI Ke-dong. (2002). The quantitative and qualitative research in the field of educational technology. Retrieved March 7, 2007, from 
Http ://w w w. iteonline.net. 

XU Jia. (2002). What is the shortage in the project research? China Education Daily, 2002, 5. 

HUANG Rong-huai. (2002). The analyses of the trend of educational technology in China. Retrieved July 26, 2007, from 
http://www.tjjy.com.en/jysd/xxjshuiyi/zhuanjia/hrh/5.htm. 

(Edited by SHI Li-fang and REN Li-ping) 



63 




